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LONG-TERM  GOALS 

The  goal  of  our  work  is  to  develop  a  system  for  optical  communications  through  the  sea-air  interface 
and  to  provide  a  predictive  performance  model  for  the  system  for  different  environmental  conditions, 
source  power  levels,  receiver  range,  etc. 

OBJECTIVES 

The  scientific  objectives  of  the  project  involve  understanding  the  spectral  and  temporal  effects  of  water 
constituents  and  surface  characteristics  on  trans-interface  communication.  This  involves  the 
development  of  a  system  that  utilizes  several,  separated,  redundant,  sources  to  reduce  the  data  drop-out 
rate  due  to  wave  refraction.  Engineering  objectives  of  the  project  are  the  design  and  development  of  an 
intelligent  optical  receiver  that  can  identify  the  positions  of  redundant  sources  within  video  imagery 
and  control  a  micro  electro-mechanical  system  (MEMS)  device  to  direct  radiance  from  only  those 
sources  to  a  high-speed  sensor  for  demodulation.  This  spatial  filtering  should  reduce  interference  from 
surface  reflection  of  skylight. 

APPROACH 

Wave  refraction  of  light  rays  passing  through  the  sea-air  interface  causes  a  perturbation  in  the  path  of 
the  ray  relative  to  its  position  and  direction  for  a  flat  sea  surface.  While  gravity  waves  range  in  slopes 
up  to  about  15°  (e.g.  Lighthill  1980),  these  slopes  can  be  augmented  near  the  crests  by  parasitic 
capillary  waves  (Martin  2004).  According  to  the  Cox-Munk  (Cox  and  Munk  1954)  wave-slope 
relationship  to  wind  speed,  a  wind  of  lOm/s  results  in  a  mean  wave  slope  of  about  0. 19.  Wu  (1990) 
suggests  a  more  rapid  increase  of  slope  with  wind  speed,  however.  Past  experience  in  trying  to  avoid 
sun  glint  when  performing  remote  sensing  operations  indicates  that  slopes  of  0.6  are  not  unusual  for 
individual  waves. 

For  capillary  waves,  the  disruption  in  directionality  of  an  optical  beam  or  ray  that  results  in 
information  dropouts  can  be  significant.  This  is  corroborated  by  an  experiment  made  in  Bayboro 
Harbor  (St.  Petersburg,  FL)  where  the  continuity  of  signal  from  a  single  LED  was  contrasted  against 
that  from  up  to  four  LEDs,  where  four  LEDs  were  separated  by  four  feet  at  the  comers  of  a  square 
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frame  approximately  1  foot  below  the  interface.  Generating  capillary  waves  with  2  air  jets  provides 
enough  capillary  wave  activity  to  provide  significant  signal  drop-outs.  Redundant,  spatially  separated 
sources  are  used  with  digital  signal  processing  (DSP)  of  video  imagery  to  identify  the  positions  of 
redundant  sources  within  the  imagery  (areas  of  interest,  AOIs).  Radiance  from  these  dispersed  sources 
it  then  collected  onto  a  common  intensified  receiver  to  reduce  interference  from  surface  reflection  of 
skylight  while  increasing  the  signal  for  high-rate  data  reception. 

BACKGROUND 

In  this  concept  for  optical  communication  through  the  air-sea  interface,  multiple  emitters  are  placed 
beneath  the  water’s  surface  and  imaged  from  above.  These  emitters  appear  as  small  points  of  light  in  a 
video  image  of  the  area.  In  a  project  funded  by  Blackbird  Technologies,  we  have  demonstrated  the 
ability  to  process  video  in  real-time  and  identify  our  emitters  within  the  video.  For  this  work,  we 
designed  optical  emitters  using  LEDs  that  transmit  data  using  on/off  keying.  Since  standard  video 
rates  are  used,  data  bandwidth  is  limited  to  5  baud  and  only  short  messages  containing  emitter  ID 
information  is  transmitted. 

Utilizing  digital  signal  processing  (DSP)  hardware,  we  developed  a  system  that  processes  NTSC  video, 
identifies  sources  within  the  video  matching  search  criteria,  decodes  data  transmitted  by  those  sources, 
and  outputs  the  input  video  signal  with  graphical  overlays  highlighting  the  detected  signals  and 
displaying  received  data.  Fig.  1  shows  the  prototype  electronics  (left)  and  a  screen  capture  from  a 
video  clip  (right).  The  screen  capture  shows  several  sources  of  light  imaged  through  a  narrow  band¬ 
pass  filter.  For  the  case  shown,  only  one  point  of  light  is  actually  a  device  (ID  399)  communicating 
data.  The  other  points  are  street  and  car  lamps  with  energy  in  the  pass  band  of  the  optical  filter. 
Without  our  DSP  approach,  these  other  points  would  not  only  decrease  S/N  but  could  be  identified  as 
spurious  targets.  We  have  demonstrated  the  ability  to  maintain  detection  of  signal  sources  under  a 
wide  range  of  environmental  conditions,  have  demonstrated  tracking  of  moving  signal  sources,  and 
proven  our  ability  to  distinguish  our  devices  from  sources  that  may  appear  similar  in  nature. 

This  extension  of  the  basic  system  involves  the  use  of  a  spatial  modulator  to  optically  direct  only  the 
portions  of  the  image  containing  signal  sources  to  high-speed  detectors  for  data  reception.  This 
concept  is  illustrated  below  in  Fig.  2.  Any  camera  outputting  NTSC  video  is  used  to  image  the  scene  of 
interest.  The  video  signal  is  processed  by  the  DSP  electronics  to  determine  the  coordinates  of  signal 
sources.  These  coordinates  are  used  to  control  an  active  modulator  to  direct  only  the  portions  of  the 
image  containing  signal  sources  to  a  high-speed  detector,  reducing  background  noise  and  raising  the 
signal-to-noise  ratio. 

MEMS  2-D  imaging  arrays  are  commercially  available  and  widely  used  in  TV  and  data  projection 
systems  [eg.  Texas  Instruments  Digital  Mirror  Device  (DMD)].  The  DMD  is  designed  to  tilt  each  of 
several  million  mirrors  into  1  of  2  positions  according  to  a  video  image  received  by  the  coupled 
electronics  module.  In  the  first  (default)  position  the  mirrors  are  turned  off  and  are  all  coplanar. 
Incoming  radiation  is  simply  reflected  at  the  same  angle  as  the  incoming  beam  (relative  to  the  surface 
nonnal). 
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Fig.  1:  Prototype  electronics  used  to  process  video  signals  to  locate  and  decode 
optical  signals,  and  a  representative  screen  capture  showing  the  detection 
of  a  signal  source  placed  near  many  noise  sources. 


In  the  second  position,  the  mirrors  are  tilted  by  approximately  7  degrees.  Depending  on  the  orientation 
of  the  DMD,  the  reflected  beam  can  either  be  at  a  larger  or  smaller  angle  (by  14  degrees)  than  the  beam 
reflecting  off  the  mirrors  in  the  first  position.  By  turning 
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Fig.  2:  Actively-steered  high-speed,  high  sensitivity  detector.  2-D  Active  Array 
(Eg.  TI DLP)  is  used  to  actively  project  only  areas  of  the  scene  of  interest  onto 
a  high  speed,  highly-sensitive  PMT  detector. 


the  appropriate  mirrors  on  or  off,  light  from  specific  portions  of  the  image  can  be  separated  from  other 
regions.  It  is  proposed  that  such  a  2D  DMD  array  be  utilized  within  a  custom  optical  system  to  allow 
simultaneous  targeting  of  each  of  the  individual  sources  that  comprise  the  underwater  transmitter. 

Only  those  mirror  elements  corresponding  to  the  locations  of  the  signals  will  be  energized  to  redirect 
the  signals  to  the  high-speed  detector.  As  wave  action  and  other  factors  interfere  with  the  reception  of 
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an  individual  source,  the  corresponding  mirror  elements  will  be  turned  off  and  only  light  from  the 
other  valid  sources  will  continue  to  the  high-speed  detector.  In  the  best  case,  all  individual  sources 
will  be  present  and  their  combined  energy  will  be  directed  to  the  detector  for  maximum  signal  to  noise. 
Note  that  by  rejecting  residual  background  image  noise  due  to  reflected  skylight  or  sun  glint,  the  gain 
on  the  high-speed  detector  can  be  increased. 

WORK  COMPLETED 

Scientific  A  single-wavelength  (525  nm),  4-source,  LED  array  was  constructed  and  deployed  in  both 
day  and  night  field  tests.  Analysis  of  the  data  acquired  during  these  deployments  indicates  that  the 
basic  hypothesis  is  correct  and  that  our  envisioned  approach  is  viable  (Carder  et  al.  2007).  Redundant 
sources  increased  transmission  efficiency  in  both  daytime  and  nocturnal  field  tests.  We  utilized  only 
narrow-band-pass  and  polarization  filters  on  the  camera.  Compared  to  losses  with  one  LED,  data  drop¬ 
outs  for  the  redundant  sources  were  reduced  by  factors  of  6-14  (see  Fig.  3). 


LED  Transmission  Efficiency  (Day) 


Fig.  3:  Statistical  analysis  of  the  various  permutations  of  the  four, 
redundant,  LED  sources.  Transmission  efficiency  increased  to 
95%  utilizing  all  four  sources  versus  30%  utilizing  a  single  LED. 


When  completed,  spatial  filtering  is  expected  to  increase  the  signal-to-noise,  and  thus  range  and/or  data 
rate,  by  a  factor  equivalent  to  the  ratio  of  the  total  image  area  divided  by  the  sum  of  the  LED- 
neighborhood  areas  of  the  images  (e.g.  >  20X). 

Engineering  Engineering  activities  have  focused  on  construction  and  testing  of  prototype  hardware 
to  demonstrate  proof-of-concept.  We  have  completed  the  initial  design  of  this  hardware,  but  have 
experienced  significant  delays  in  construction  and  testing.  Both  purchasing  and  equipment 
malfunctions  have  delayed  the  project  by  about  four  months.  Work  is  progressing  well  now  and  we 
expect  to  complete  this  project  within  budget.  A  summary  of  work  completed  is  provided  below. 
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We  completed  construction  of  the  prototype  receiving  hardware  as  shown  below  in  Fig.  4.  The  main 
lens  views  the  target  and  produces  an  image  on  a  beam  splitter.  One  path  from  this  beam  splitter  is 
directed  towards  a  video  camera  for  image  display  and  processing.  In  the  second  path,  an  enlarging 
lens  relays  the  primary  image  onto  the  spatial  modulator  (DMD).  To  compensate  for  the  different 
sizes  of  the  camera  sensor  and  the  spatial  modulator,  the  relayed  image  is  magnified  by  a  factor  of  1.6. 
Light  reflecting  off  the  spatial  modulator  is  collected  and  focused  onto  the  data  detector  for  subsequent 
analysis. 


Fig.  4:  A  photo  of  the  prototype  optical  receiving  hardware. 

Not  shown  in  the  figure  are  the  electronics  used  to  process  images  and  control  the  spatial  modulator 
(refer  to  Fig.  1).  In  our  concept,  optical  transmitters  will  appear  as  small  dots  in  the  images.  These 
transmitters  will  modulate  data  at  a  high  rate  (>  100  Khz)  within  a  much  slower  modulating  envelope 
(<  10  Hz)  that  can  be  detected  with  a  standard  video  camera.  Image  processing  will  detect  the  slow 
modulation  and  activate  only  those  elements  of  the  spatial  modulator  upon  which  the  modulating 
signal  is  incident.  The  active  elements  reflect  the  incident  light  to  the  data  (high-speed)  detector  for 
analysis.  The  goal  of  this  approach  is  to  increase  the  system’s  signal-to-noise  ratio  by  filtering  out 
ambient  scene  light  and  ultimately  detecting  only  that  light  received  from  the  transmitter(s). 

To  increase  gain,  we  replaced  the  photodiode  initially  chosen  for  the  data  detector  with  a 
photomultiplier  tube  (PMT).  These  detectors  offer  excellent  sensitivity  and  frequency  response  at  the 
cost  of  complexity.  There  was  an  unanticipated  delay  of  90  days  waiting  for  the  manufacturer  to  ship 
this  detector.  While  software  development  could  somewhat  continue  during  this  time,  hardware 
development  essentially  was  on  hold.  We  have  received  this  detector  and  are  currently  integrating  it 
into  the  system. 
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We  have  completed  initial  programming  and  evaluation  of  the  software  required  to  operate  the 
receiving  system.  In  our  approach,  the  software  contains  nine  fundamental  modules: 

1 .  Camera  capture  thread 

2.  Frame  hank  switching 

3.  Optical  communication  interface 

4.  Bitmap  stream  creation 

5.  Display  thread 

6.  DMD  activeX  control 

7.  DMD  communication 

8.  System  timing/control 

9.  Graphical  user  interface  (GUI) 

Each  of  the  nine  modules  have  been  programmed  and  passed  initial  testing.  Two  challenges  in  this 
project  have  been  the  integration  of  the  all  modules  under  the  Windows  operating  system  and  a  longer- 
than-expected  communication  time  to  the  DMD  device.  The  module  integration  issues  have  been 
addressed  by  investigating  their  operation  separately  and  the  full  application  is  currently  being 
reassembled.  The  communication  delays  appear  to  be  caused  by  a  slow  universal  serial  bus  (USB) 
implementation  on  the  DMD  commercial  hardware.  While  this  means  we  will  not  be  able  to  update 
the  DMD  at  full  video  rates  (30  Hz),  it  should  not  adversely  affect  the  overall  system’s  performance. 
Since  our  initial  demonstrations  will  utilize  stationary  emitters,  we  do  not  require  rapid  adjustment  of 
the  DMD. 

Work  over  the  remainder  of  the  project  will  include: 

•  Complete  integration  of  the  PMT 

•  Laboratory  testing  and  characterization  of  the  receiving  hardware/software 

•  Out-door  demonstration  of  the  system. 

RESULTS 

•  Use  of  four  redundant  LED  light  sources  separated  by  ~1  m  decreased  the  data  drop-outs  from 
as  much  as  -70%  to  -5%  for  a  daylight  test  across  a  capillary-wave  interface 

•  We  have  completed  initial  programming  and  evaluation  of  the  software  required  to  operate  the 
receiving  system:  nine  fundamental  modules: 

o  Camera  capture  thread 
o  Frame  bank  switching 
o  Optical  communication  interface 
o  Bitmap  stream  creation 
o  Display  thread 
o  DMD  activeX  control 
o  DMD  communication 
o  System  timing/control 
o  Graphical  user  interface  (GUI) 

•  All  components  are  available  for  integration  and  testing 
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IMPACT/APPLICATIONS 


There  are  many  scenarios  where  communication  between  a  submerged  asset  and  an  airborne  asset 
would  be  very  desirable.  However,  communication  through  the  air/sea  interface  is,  at  best,  problematic 
and  any  high-bandwidth  communication  strategy  remains  elusive.  Field  tests  of  our  basic  approach 
(not  yet  with  spatial  filtering)  have  shown  that  our  hypothesis  is  sound,  and  data  dropouts  were 
reduced  by  factors  from  6X  to  14X.  Additional,  brighter  LED  sources  will  further  reduce  drop-outs. 
With  the  addition  of  the  DSP  and  MEMS  DMD  technology,  we  expect  further  rejection  of  ambient 
noise  and  an  increase  in  S/N  (~20X)  and  data  rate.  This  technology  should  not  only  make  optical 
communication  through  the  air/sea  interface  feasible  but  also  allow  for  high-bandwidth 
communication. 

RELATED  PROJECTS 

•  USF  College  of  Marine  Science  Center  for  Underwater  Observability  and  Optical 
Communication  -  Utilization  of  the  ROSEBUD  ROV  and  the  TOPO-13  air  platform 

•  Eric  Kaltenbacher,  SRI  International  -  Optical  tagging  and  3D  underwater  imaging. 
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